Managing challenge events

ABSTRACT

A method and system for managing a challenge event. At least two participants are selected to participate in the challenge event, in response to receipt of a request for the challenge event from a client computer. Assets of software code are provided to the participants for assisting the participants to deliver respective submissions as outcomes of participation by the participants in the challenge event. A solution to the challenge event in a form of a computer code is received from only one participant, of the at least two participants, previously chosen by from the selected at least two participants as the only participant to submit the solution.

This application is a continuation application claiming priority to Ser.No. 14/455,974, filed Aug. 11, 2014, now U.S. Pat. No. 9,327,198, issuedMay 3, 2016, which is a continuation of Ser. No. 13/013,478, filed onJan. 25, 2011, U.S. Pat. No. 9,199,172, issued Dec. 1, 2015.

BACKGROUND

The present application relates to the field of computers, andspecifically to the use of computers in management of software design,coding and other work events, normally used in building softwareapplications. Even more particularly, the present invention relates tothe use of computers in creating events, scoring submitted designs andcode, and rewarding event participants for submissions that meet anacceptance criteria.

BRIEF SUMMARY

A computer implemented apparatus and computer program product providesdesign, coding, and other work challenge events and competition eventsassociated with building a software application. The apparatus includesa portal for communicating the events to potential participants andreceives computer software designs and code submissions from selectedparticipants. An event manager server creates the events. The apparatusincludes a catalog of reusable code assets and a pricing calculator onthe event manager server for pricing the event based on use of thecatalog and based on results of previous events as determined by ananalytics engine. Participants are selected using a selector subsystem.Computer code submitted by the selected participants is executed in achallenge server layer and also evaluated and scored. A design event mayrequire creation of a formal document or use of a design tool such aRational Software Architect available from International BusinessMachines Corporation of Armonk, N.Y. A payment application rewardsparticipants according to the event specifications and the scoringresults or other outcome meets the event specifications and a minimumthreshold scoring result.

A major benefit of the present invention is that challenge andcompetition events are completed in short cycles. Research has shownthat work that is properly broken down to a consumable level can becompleted with the highest chance of success. Although, any cycle timemay be used with the present invention, typically events are conductedin short cycles, usually of three to seven days duration using workspecifications that have been broken down to a consumable level.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 depicts an exemplary computer in which the present invention maybe implemented;

FIGS. 2a and 2b are block diagrams showing the progression of achallenge event and a competition event respectively; and

FIG. 3 is a block diagram of one embodiment of the present invention.

DETAILED DESCRIPTION

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system or computer program product.Accordingly, aspects of the present invention may take the form of anentirely hardware embodiment, and entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as an “apparatus” “module” or “system.”Furthermore aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablestorage mediums(s) having computer readable program code embodiedthereon.

A computer readable storage medium may be, for example, but not limitedto, an electronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus, or device, or any suitable combinationof the foregoing. More specific examples (a non-exhaustive list) of thecomputer readable storage medium would include the following: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only (EPROM or Flash memory), aportable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

With reference now to the figures, and in particular to FIG. 1, there isdepicted a block diagram of an exemplary computer 102, which may beutilized by the present invention. Note that some or all of theexemplary architecture, including both depicted hardware and software,shown for and within computer 102 may be utilized by software deployingserver 150.

Computer 102 includes a processor 104 that is coupled to a system bus106. Processor 104 may utilize one or more processors, each of which hasone or more processor cores. A video adapter 108, which drives/supportsa display 110, is also coupled to system bus 106. System bus 106 iscoupled via a bus bridge 112 to an input/output (I/O) bus 114. An I/Ointerface 116 is coupled to I/O bus 114. I/O interface 116 affordscommunication with various I/O devices, including a keyboard 118, amouse 120, a media tray 122 (which may include storage devices such asCD-ROM drives, multi-media interfaces, etc.), a printer 124, andexternal USB port(s) 126. While the format of the ports connected to I/Ointerface 116 may be any known to those skilled in the art of computerarchitecture, in one embodiment some or all of these ports are universalserial bus (USB) ports.

As depicted, computer 102 is able to communicate with a softwaredeploying server 150 using a network interface 130. Network 128 may bean external network such as the Internet, or an internal network such asan Ethernet or a virtual private network (VPN).

A hard drive interface 132 is also coupled to system bus 106. Hard driveinterface 132 interfaces with a hard drive 134. In one embodiment, harddrive 134 populates a system memory 136, which is also coupled to systembus 106. System memory is defined as a lowest level of volatile memoryin computer 102. This volatile memory includes additional higher levelsof volatile memory (not shown), including, but not limited to, cachememory, registers and buffers. Data that populates system memory 136includes computer 102's operating system (OS) 138 and applicationprograms 144.

OS 138 includes a shell 140, for providing transparent user access toresources such as application programs 144. Generally, shell 140 is aprogram that provides an interpreter and an interface between the userand the operating system. More specifically, shell 140 executes commandsthat are entered into a command line user interface or from a file.Thus, shell 140, also called a command processor, is generally thehighest level of the operating system software hierarchy and serves as acommand interpreter. The shell provides a system prompt, interpretscommands entered by keyboard, mouse, or other user input media, andsends the interpreted command(s) to the appropriate lower levels of theoperating system (e.g., a kernel 142) for processing. Note that whileshell 140 is a text-based, line-oriented user interface, the presentinvention will equally well support other user interface modes, such asgraphical, voice, gestural, etc.

As depicted, OS 138 also includes kernel 142, which includes lowerlevels of functionality for OS 138, including providing essentialservices required by other parts of OS 138 and application programs 144,including memory management, process and task management, diskmanagement, and mouse and keyboard management.

Application programs 144 include a renderer, shown in exemplary manneras a browser 146. Browser 146 includes program modules and instructionsenabling a world wide web (WWW) client (i.e., computer 102) to send andreceive network messages to the Internet using hypertext transferprotocol (HTTP) messaging, thus enabling communication with softwaredeploying server 150 and other computer systems.

Application programs 144 in computer 102's system memory (as well assoftware deploying server 150's system memory) also include a portal148. Portal 148 includes code for implementing the processes describedbelow, including those described in FIGS. 2-3. In one embodiment,computer 102 is able to download portal 148 from software deployingserver 150, including in an on-demand basis, wherein the code in portal148 is not downloaded until needed for execution to define and/orimplement the invention described herein. In other embodiments, theportal is a web based application that is accessed from a clientcomputer over a network using a web browser. Note further that, in oneembodiment of the present invention, software deploying server 150performs all of the functions associated with the present invention,thus freeing computer 102 from having to use its own internal computingresources to execute portal 148.

The hardware elements depicted in computer 102 are not intended to beexhaustive, but rather are representative to highlight essentialcomponents required by the present invention. For instance, computer 102may include alternate memory storage devices such as magnetic cassettes,digital versatile disks (DVDs), Bernoulli cartridges, and the like.These and other variations are intended to be within the spirit andscope of the present invention.

In FIG. 2a , there is shown a block diagram depicting the progression ofsteps 210-222 performed during a challenge event.

In step 210 a detailed description of a challenge event is posted on aportal. A challenge event is also referred to as a component herein, andthe detailed description is also referred to as a specification herein.The specification may include use of certain reusable code assets from acatalog posted on the portal. The specification also includes theresults of a pricing calculator described below from an event managerserver.

In step 212, participants (also referred to herein as vendor players)propose their respective solutions and terms such as cost, schedule, orapproaches to provide a design or code their solution or provide anotheroutcome based on the selected event. Participants may accept the pricein the specification or propose a higher or lower price. Other termsfrom the specification may also be changed in their proposal. A singleparticipant is then chosen for each challenge event using a selectorsubsystem (not shown). An important consideration in making theselection is each potential participant's digital reputation. In everyprevious event, participants are evaluated on reliability, i.e., didthey submit and submit on time? They are scored on adherence tospecifications using a standard scorecard with a 0-100 rating scale.Additional elements are also scored, including frequency of reuse fromthe catalog, the type of events the participant is working on, and howrecent their event history is. All of these elements become part of theparticipant's digital reputation which is continuously maintained,updated, and available for use by the selector subsystem. Reliability ofboth submissions and wins may be displayed by month as a time line graphin a digital reputation display. The digital reputation of participantsmay be stored in a database accessible by the challenge server layer.After producing the work, the chosen single participant then delivershis solution in step 214.

In step 216, a challenge server layer in communication with the portal,evaluates submitted computer code. For example, a static analysis ofsubmitted computer code may be performed. The code is validated forquality and evaluated using objective criteria, such as a publishedscorecard. If fixes are required, the participant applies the fixes instep 218. This process can iterate. Once the outcome (the submitteddesign or code) meets the specification and is validated for quality, itis accepted and is delivered to the client who originally requested thecoding event.

In step 220, compensation is distributed to the participant. Thesubmitted code is added to the catalog of reusable assets. Compensationmay be a payment in actual dollars (green dollars), a credit of billablehour, digital reputation credit, or any other type of compensation knownin the art and agreed to by the participant at the time the participantis chosen for the challenge coding. The client is then billed for thechallenge event in step 222.

In FIG. 2b , the steps of a competition event are shown. In step 250, adetailed description (specification) of a competition event (component)is posted to the same portal used for challenge events as describedabove. The results from the same pricing calculator described above areincluded in the competition event specification.

In step 252, approved participants (company staff) register to providetheir solutions to the specification. Some or all of the approvedparticipants may be the same individuals who participant in challengeevent as described above. Other limitations may be placed on who may beparticipants, for example, participant may be limited to employees of aparticular company or employees who are temporarily unassigned to otherprojects. The approved participants deliver their solutions according toa time schedule in the specification, in step 254.

In step 256, each of the submitted code solutions for a competition isevaluated by performing a static analysis of the submitted computercode. The quality is validated and each solution is scored using apublished scorecard. A single best solution is selected as the winner ofthe competition. A second place winner may also be selected. Otherwinners may also be selected according to the competition eventdescription.

If fixes are needed they are applied in step 258 and the winningsoftware design or computer code is delivered to the client whorequested the competition be held.

Compensation is distributed in step 260. If more than one winner isselected, then more than one participant my receive compensation. Asabove, the compensation may be in the form of green dollars, billablehours credit, or any other form of compensation. The first place winingsolution is added to the catalog of reusable assets.

In step 262, the client is billed for the outcome (first place winningsoftware design or computer code).

In FIG. 3, there is shown a system block diagram of the presentinvention. An engagement team 301 operates the system on behalf of oneor more clients. The system includes an event manager server forcreating the events. The engagement team uses block 303 to forecastdemand for events and manage capacity by sourcing and supplying staff304 for managing events. This may include procuring channel partners, aswell as registering participants from suppliers 302, for both challengeevents and competition events in block 324. Outcomes are delivered toclients by engagement team 301 from block 318.

Rules for event specifications are developed and event staff trainedusing blocks 305 and 306 respectively.

A catalog of reusable assets is structured in block 307. After eachevent, the delivered solution is added to the catalog in block 308.Other code assets may be added in block 309.

A project comprising one or more events is registered by block 310. Theevent includes the results of pricing calculator 311 described below.The requirements in the specification of an event are validated in block312. However, block 312 may be bypassed by an event leader who iscertified to operate events without assistance.

An event proceeds by dispatching a work request in block 313 and isexecuted in block 314 by event manager 323. In block 315, solutions areexecuted, evaluated for quality, and scored.

Selector subsystem 316 selects a winner for competition events. Finalaccounting, including compensating the participants(s) and billing theclient, is preformed in block 317. The solution is delivered to theclient using block 318 by engagement team 301.

The performance of the entire system is continually improved bygathering market intelligence in block 319 and by monitoring eventperformance in block 320. Additional event types may be developed inblock 321. The event platforms, including the portal, the pricingcalculator, and catalog, are managed in block 322. In block 324, overallperformance of the system is analyzed and optimized.

In block 325, the community of participants is promoted. In block 326,the reputation of the participants is managed and its variouscompensation methods reviewed for improvement.

Pricing calculator 311 is an integrated capability of the event managerserver that allows the client or event sponsor to enter characteristicsabout the event, such as the number of components, use case scenarios,or screens, and receive recommendations for sizing the event based onanalytics captured from previous events. Pricing calculator 311 providesa total price for running the event based on size and components fromreusable asset catalog 309. Correctly sizing an event is very importantto getting participants to register to work on the event.

As noted above, blocks 303 and 304 are important elements and areimportant features in managing events. A recommendation engine in theportal uses business analytics techniques to manage supply and demand,and to make recommendations for current and future events. Some of thedata elements which may be captured by the recommendation engineinclude:

-   -   event type and technology platform    -   event duration    -   event value in points, hours, and dollars    -   assets specified    -   number of business rules, use cases, classes, or objects in        event specification    -   event registrants and digital reputations    -   submission scores    -   assets reused by participants    -   scores of delivered solutions        The recommendation engine uses this comprehensive outcome level        data with business analytics techniques to provide:    -   recommendations of participants who would be expected to perform        well in an event    -   recommendations for participants of events that they should        consider registering for    -   recommendations for participants of assets that may help them        deliver an outcome more efficiently    -   success prediction of likelihood of a successful outcome based        on event parameters    -   identifying changes to skills or resources 304 needed to meet        forecasted demand for outcomes    -   alerts for troubled events        Various business analytics techniques known in the art may be        used such as, but not limited to, computer algorithms in which        the parameters are continuously adjusted based on current        outcomes, or static models developed from historical data.

The present invention is also used to produce other types of outcomes tosupport the software development and support lifecycle. These otheroutcomes are required elements on projects where software code anddesign is being produced, and therefore, the computer system (portalapplication) supports the delivery of these additional accessory outcometypes in order to assemble a working software system. The portalapplication allows for a selection on event type, with multiple choicesfor the different types of software project outcomes represented. Theportal allows for outcomes to be requested in the areas of test casecreation, test case execution, system architecture deliverables, graphicdesigns, software component assembly, and idea generation. The scoringmethod for these types of outcomes is consistent with coding and designevents, but the scorecard used and the evaluation criteria are specificto the type of outcome.

While there have been shown and described what are at present consideredthe preferred embodiments of the invention, it will be obvious to thoseskilled in the art that various changes and modifications may be madetherein without departing from the scope of the invention as defined bythe appended claims. For example, it will be obvious that some stepsshown sequentially may be performed in parallel.

What is claimed is:
 1. A method for obtaining, from managing a challengeevent, a computer code that supports software development for a clientcomputer via use of a computer system comprising an apparatus, whereinthe apparatus comprises a processor and a read-only memory (ROM)hard-wired into the apparatus, wherein the ROM contains program code,which, upon being executed by the processor, implements the method, andwherein the apparatus is a special purpose machine specific to themethod due to the ROM being hard wired into the apparatus, said methodcomprising: in response to the processor receiving a request for thechallenge event from a client computer, said processor selecting aplurality of participants to participate in the challenge event; saidprocessor providing to the participants assets of software code forassisting the participants to deliver respective submissions as outcomesof participation by the participants in the challenge event; saidprocessor receiving a solution to the challenge event in a form of acomputer code from only one participant, of the plurality ofparticipants, previously chosen by the processor from the selectedplurality of participants as the only participant to submit thesolution; said processor validating the received solution for qualityand accepting the received solution; and in response to said validatingand accepting the received solution, said processor delivering thereceived solution to the client computer, said received solutionconfigured to be used by the client computer to support softwaredevelopment for the client computer.
 2. The method of claim 1, whereinsaid selecting the participants comprises utilizing data elements, andwherein the data elements comprise an event type and technology platformof the challenge event, a duration of the challenge event, and anumerical value of the challenge event.
 3. The method of claim 1, sadmethod comprising: said processor forecasting a demand for outcomesresulting from the challenge event; and said processor identifyingchanges needed to meet the forecasted demand, wherein the changes areselected from the group consisting of changes to resources, changes toskills, and combinations thereof.
 4. The method of claim 1, wherein themethod comprises: said processor receiving from each participant of theselected plurality of participants a proposal, for solving the challengeevent, that meets an event specification, wherein the only oneparticipant was chosen based on the proposal received from eachparticipant of the selected plurality of participants.
 5. The method ofclaim 1, wherein the challenge event is a present challenge event,wherein the only one participant was chosen by determining a digitalreputation score for each participant and choosing the only oneparticipant having the highest digital reputation score, and wherein thedigital reputation score for each participant is based on eachparticipant's performance on previous challenge events that preceded thepresent challenge event.
 6. The method of claim 5, wherein the digitalreputation score for each participant includes a score for: (i) eachparticipant's frequency of reuse of the reusable code assets in acatalog of reusable code assets for the previous challenge events, (ii)the type of the previous challenge events that each participant workedon, and (iii) how recent the previous challenge events are for eachparticipant.
 7. A computer program product, comprising a read-onlymemory (ROM) hard-wired into an apparatus that includes a processor,said ROM having computer readable program code stored therein, saidprogram code containing instructions which, upon being executed by theprocessor, implement a method for obtaining, from managing a challengeevent, a computer code that supports software development for a clientcomputer, said apparatus being a special purpose machine specific to themethod due to the ROM being hard wired into the apparatus, said methodcomprising: in response to the processor receiving a request for thechallenge event from a client computer, said processor selecting aplurality of participants to participate in the challenge event; saidprocessor providing to the participants assets of software code forassisting the participants to deliver respective submissions as outcomesof participation by the participants in the challenge event; saidprocessor receiving a solution to the challenge event in a form of acomputer code from only one participant, of the plurality ofparticipants, previously chosen by the processor from the selectedplurality of participants as the only participant to submit thesolution; said processor validating the received solution for qualityand accepting the received solution; and in response to said validatingand accepting the received solution, said processor delivering thereceived solution to the client computer, said received solutionconfigured to be used by the client computer to support softwaredevelopment for the client computer.
 8. The computer program product ofclaim 7, wherein said selecting the participants comprises utilizingdata elements, and wherein the data elements comprise an event type andtechnology platform of the challenge event, a duration of the challengeevent, and a numerical value of the challenge event.
 9. The computerprogram product of claim 7, sad method comprising: said processorforecasting a demand for outcomes resulting from the challenge event;and said processor identifying changes needed to meet the forecasteddemand, wherein the changes are selected from the group consisting ofchanges to resources, changes to skills, and combinations thereof.
 10. Acomputer system comprising an apparatus, said apparatus comprising aprocessor and a read-only memory (ROM) hard-wired into the apparatus,said ROM containing program code which, upon being executed by theprocessor implements a method obtaining, from managing a challengeevent, a computer code that supports software development for a clientcomputer, said apparatus being a special purpose machine specific to themethod due to the ROM being hard wired into the apparatus, said methodcomprising: in response to the processor receiving a request for thechallenge event from a client computer, said processor selecting aplurality of participants to participate in the challenge event; saidprocessor providing to the participants assets of software code forassisting the participants to deliver respective submissions as outcomesof participation by the participants in the challenge event; saidprocessor receiving a solution to the challenge event in a form of acomputer code from only one participant, of the plurality ofparticipants, previously chosen by the processor from the selectedplurality of participants as the only participant to submit thesolution; said processor validating the received solution for qualityand accepting the received solution; and in response to said validatingand accepting the received solution, said processor delivering thereceived solution to the client computer, said received solutionconfigured to be used by the client computer to support softwaredevelopment for the client computer.
 11. The computer system of claim10, wherein said selecting the participants comprises utilizing dataelements, and wherein the data elements comprise an event type andtechnology platform of the challenge event, a duration of the challengeevent, and a numerical value of the challenge event.